The present invention relates generally to the field of microscopy and in particular to a new and useful apparatus and method for high-speed image scanning, image capturing, and mosaicking to enlarge the field of view with respect to microscopic objects.
A microscope is an indispensable tool for micro-assembly and micro-manipulation. However, conventional microscopes suffer from the limitation that high magnification reduces the size of the field of view, which is the maximum object size which may be imaged by a lens. As a result, many micro-assembly or micro-manipulation tasks that require micron to sub-micron precision over millimeter work volume are beyond the capability of fixed optical microscopes.
For example, in vitro fertilization requires the manipulation of two microscopic biological cells, a spermatazoid and an ovule. The two biological cells may be separately located in two different zones of interest on a biological plate. If a first zone of interest is magnified for viewing and manipulation of one biological cell, a biological cell located at a distant second zone of interest may fall outside the field of view of the first zone of interest. This is especially problematic where the biological cell at the second zone of interest is moving.
Another example involves micro-assembly by mobile robot using relative positioning. Stick-and-slip microrobots for instance offer a very high relative accuracy (a few nm) in a large working space. However, high resolution sensors that can work on a large scale are usually expensive and volume-consuming. Moreover, one sensor is required for each degree of freedom. One way to reduce the number of sensors is to use multi-dimensional sensors. Using a pattern-matching algorithm, it is possible to track the motion of an object via a CCD-camera looking through a microscope.
The accuracy of this sensing method depends on magnification but an accuracy of half a micron to a quarter of micron can be reasonably achieved. X and Y movement and rotation can be sensed without defocusing the microscope and the Z position can be obtained by focusing-defocusing the picture. However, the working volume will be limited to the size of the picture itself which is a problem if accuracy is required. If the robot has to perform a task like a pick-and-place manipulation this volume may not be large enough. The whole assembly setup might have to be moved under the microscope which may be a problem if delicate assembly is required.
A common solution to the problem of reduced field of view is to move the platform supporting the sample or to move the microscope itself. The bandwidth of the motion is limited by the inertia of the platform or microscope, and the vibration resulting from the motion can blur the image or even modify the scene.
Mosaicking, or forming a single large image from smaller images, is used in applications such as NASA planetary flybys and photo-stitching software in some consumer digital cameras. However, implementation of mosaicking for performance of dynamic micro-assembly and micro-manipulation tasks with real-time vision guidance requires an optical system with a sufficiently fast refresh rate. Confocal microscopes employ high speed scanning to form images but only a single pixel data is obtained at each scan.
Similarly, U.S. Pat. No. 6,433,907 to Lippert et al. teaches a display apparatus that includes a scanning assembly that scans a plurality of light beams produced from spatially distinct regions, in a raster pattern. The scanning assembly includes mirrors that pivot to sweep the beams. Beam color and intensity is modulated to form a respective pixel of an image. By properly controlling the color and intensity of the beam for each pixel location, the display can produce a contiguous image from the pixels from each distinct region. Like confocal microscopes, the Lippert '907 apparatus involves acquisition of single pixel data, or pixel-by-pixel scanning.
U.S. Pat. No. 6,313,452 to Paragrano et al. discloses a microscopy system that utilizes a plurality of images to create a single mosaic image. The system comprises a stage, at least one magnifying lens, a lens controller, a video capture device, and a processing subsystem. However, no high-speed scanning or capture devices are included.
U.S. Pat. Nos. 6,101,265 and 6,226,392 both to Bacus et al. teach an apparatus and method for acquiring and storing multiple images from a specimen via a microscope and digital scanner, and providing a user a reconstructed image of the entire specimen at low magnification. The reconstructed image is formed of a large number of tiled images which are coordinated and assembled to form the reconstructed image. High-speed scanning, imaging, and refreshing are not taught.
U.S. Pat. No. 6,272,235 also to Baccus et al. further teaches that acquired images are coherently seemed together to provide virtual digitized images at either at low or high resolution. A data structure is formed with the virtual digitized images along with their mapping coordinates. The data structure is formed with compressed data so that it can be transmitted over low bandwidth channels, such as the Internet, without loss of resolution.
An optical system is needed that addresses the field of vision limitations of conventional microscopes without movement of the microscope stage or sample. The optical system should be capable of capturing images at fast refresh rates so that a virtual reconstructed image can be constructed quickly, where a view of the reconstructed image cannot be differentiated from the specimen view by the human eye. Such an optical system thereby overcomes the disadvantages of traditional motorized stages which are significantly slower. The optical system should be capable of scanning images which are focused and undistorted. Image processing in stitching the images together should be performed without any particular imaging algorithms.